Visual Illusion Created by a Striped Pattern through Augmented Reality for the Prevention of Tumbling on Stairs.

A fall on stairs can be a dangerous accident. An important indicator of falling risk is the foot clearance, which is the height of the foot when ascending stairs or the distance of the foot from the step when descending. We developed an augmented reality system with a holographic lens using a visual illusion to improve the foot clearance on stairs. The system draws a vertical striped pattern on the stair riser as the participant ascends the stairs to create the illusion that the steps are higher than the actual steps, and draws a horizontal striped pattern on the stair tread as the participant descends the stairs to create the illusion of narrower stairs. We experimentally evaluated the accuracy of the system and fitted a model to determine the appropriate stripe thickness. Finally, participants ascended and descended stairs before, during, and after using the augmented reality system. The foot clearance significantly improved, not only while the participants used the system but also after they used the system compared with before.

Tactile Transfer Learning and Object Recognition With a Multifingered Hand Using Morphology Specific Convolutional Neural Networks.

Multifingered robot hands can be extremely effective in physically exploring and recognizing objects, especially if they are extensively covered with distributed tactile sensors. Convolutional neural networks (CNNs) have been proven successful in processing high dimensional data, such as camera images, and are, therefore, very well suited to analyze distributed tactile information as well. However, a major challenge is to organize tactile inputs coming from different locations on the hand in a coherent structure that could leverage the computational properties of the CNN. Therefore, we introduce a morphology-specific CNN (MS-CNN), in which hierarchical convolutional layers are formed following the physical configuration of the tactile sensors on the robot. We equipped a four-fingered Allegro robot hand with several uSkin tactile sensors; overall, the hand is covered with 240 sensitive elements, each one measuring three-axis contact force. The MS-CNN layers process the tactile data hierarchically: at the level of small local clusters first, then each finger, and then the entire hand. We show experimentally that, after training, the robot hand can successfully recognize objects by a single touch, with a recognition rate of over 95%. Interestingly, the learned MS-CNN representation transfers well to novel tasks: by adding a limited amount of data about new objects, the network can recognize nine types of physical properties.

Single Camera Face Position-Invariant Driver's Gaze Zone Classifier Based on Frame-Sequence Recognition Using 3D Convolutional Neural Networks.

Estimating the driver's gaze in a natural real-world setting can be problematic for different challenging scenario conditions. For example, faces will undergo facial occlusions, illumination, or various face positions while driving. In this effort, we aim to reduce misclassifications in driving situations when the driver has different face distances regarding the camera. Three-dimensional Convolutional Neural Networks (CNN) models can make a spatio-temporal driver's representation that extracts features encoded in multiple adjacent frames that can describe motions. This characteristic may help ease the deficiencies of a per-frame recognition system due to the lack of context information. For example, the front, navigator, right window, left window, back mirror, and speed meter are part of the known common areas to be checked by drivers. Based on this, we implement and evaluate a model that is able to detect the head direction toward these regions having various distances from the camera. In our evaluation, the 2D CNN model had a mean average recall of 74.96% across the three models, whereas the 3D CNN model had a mean average recall of 87.02%. This result show that our proposed 3D CNN-based approach outperforms a 2D CNN per-frame recognition approach in driving situations when the driver's face has different distances from the camera.

Tool-Use Model to Reproduce the Goal Situations Considering Relationship Among Tools, Objects, Actions and Effects Using Multimodal Deep Neural Networks.

We propose a tool-use model that enables a robot to act toward a provided goal. It is important to consider features of the four factors; tools, objects actions, and effects at the same time because they are related to each other and one factor can influence the others. The tool-use model is constructed with deep neural networks (DNNs) using multimodal sensorimotor data; image, force, and joint angle information. To allow the robot to learn tool-use, we collect training data by controlling the robot to perform various object operations using several tools with multiple actions that leads different effects. Then the tool-use model is thereby trained and learns sensorimotor coordination and acquires relationships among tools, objects, actions and effects in its latent space. We can give the robot a task goal by providing an image showing the target placement and orientation of the object. Using the goal image with the tool-use model, the robot detects the features of tools and objects, and determines how to act to reproduce the target effects automatically. Then the robot generates actions adjusting to the real time situations even though the tools and objects are unknown and more complicated than trained ones.

Investigation on image signal receiving performance of photodiodes and solar panel detectors in an underground facility visible light communication system.

For the safety, underground facilities are required to be inspected regularly, especially with image analysis. Traditional wireless and wired transmission techniques have a weakness of limited transmission range in narrow underground environments. In this study, a new image transmission method based on visible light communication (VLC) has been thus proposed. Two types of detectors as an image signal receiver have been tested and discussed in the following experiments. The photodiodes (PDs) are widely used as a common image signal detector in VLC technology, but image signal detection using solar panels (SPs) has not been studied. PDs have a higher sensitivity and faster response time but a limited detection area and high cost. Besides, PDs require the lens to focus light. On the other hand, SPs have much larger optical signal receiving areas and stronger optical signal capture capabilities. They can realize lens-free detection and are inexpensive. These features of PD were firstly verified in experiments with several receiving areas and angles of detectors. The experimental result revealed that PD had better image detection and recovery capabilities than those of SP. Then, we found that a larger receiving area obtained by using double PDs/SPs improved the brightness of the restored image. In a supplementary experiment, the influence of different RGB optical components on VLC, especially the VLC-based image transmission, has been investigated by using two-dimensional Fourier transform frequency analysis. We found that the red optical component significantly increased the intensity and energy of the restored image as the image low-frequency signals were larger than the restored image using ordinary mixed white light, and moreover, the blue optical component decreased the low-frequency part of the image.

Repeated exposure to tripping like perturbations elicits more precise control and lower toe clearance of the swinging foot during steady walking.

Controlling minimum toe clearance (MTC) is considered an important factor in preventing tripping. In the current study, we investigated modifications of neuro-muscular control underlying toe clearance during steady locomotion induced by repeated exposure to tripping-like perturbations of the right swing foot. Fourteen healthy young adults (mean age 26.4 ± 3.1 years) participated in the study. The experimental protocol consisted of three identical trials, each involving three phases: steady walking (baseline), perturbation, and steady walking (post-perturbation). During the perturbation, participants experienced 30 tripping-like perturbations at unexpected timing delivered by a custom-made mechatronic perturbation device. The temporal parameters (cadence and stance phase%), mean, and standard deviation of MTC were computed across approximately 90 strides collected during both baseline and post-perturbation phases, for all trials. The effects of trial (three levels), phase (two levels: baseline and post-perturbation) and foot (two levels: right and left) on the outcome variables were analyzed using a three-way repeated measures analysis of variance. The results revealed that exposure to repeated trip-like perturbations modified MTC toward more precise control and lower toe clearance of the swinging foot, which appeared to reflect both the expectation of potential forthcoming perturbations and a quicker compensatory response in cases of a lack of balance. Moreover, locomotion control enabled subjects to maintain symmetric rhythmic features during post-perturbation steady walking. Finally, the effects of exposure to perturbation quickly disappeared among consecutive trials.

Gait Phase Detection Based on Muscle Deformation with Static Standing-Based Calibration.

Gait phase detection, which detects foot-contact and foot-off states during walking, is important for various applications, such as synchronous robotic assistance and health monitoring. Gait phase detection systems have been proposed with various wearable devices, sensing inertial, electromyography, or force myography information. In this paper, we present a novel gait phase detection system with static standing-based calibration using muscle deformation information. The gait phase detection algorithm can be calibrated within a short time using muscle deformation data by standing in several postures; it is not necessary to collect data while walking for calibration. A logistic regression algorithm is used as the machine learning algorithm, and the probability output is adjusted based on the angular velocity of the sensor. An experiment is performed with 10 subjects, and the detection accuracy of foot-contact and foot-off states is evaluated using video data for each subject. The median accuracy is approximately 90% during walking based on calibration for 60 s, which shows the feasibility of the static standing-based calibration method using muscle deformation information for foot-contact and foot-off state detection.

A Multimodal, Adjustable Sensitivity, Digital 3-Axis Skin Sensor Module.

This paper presents major improvements to a multimodal, adjustable sensitivity skin sensor module. It employs a geomagnetic 3-axis Hall effect sensor to measure changes in the position of a magnetic field generated by an electromagnet. The electromagnet is mounted on a flexible material, and different current values can be supplied to it, enabling adjustments to the sensitivity of the sensor during operation. Capacitive sensing has been added in this iteration of the module, with two sensing modalities: "pre-touch" detection with proximity sensing and normal force capacitive sensing. The sensor has been designed to be interconnected with other sensor modules to be able to cover large surfaces of a robot with normal and shear force sensing and object proximity detection. Furthermore, this paper introduces important size reductions of the previous sensor design, calibration results, and further analysis of other sensor characteristics.

A Preliminary Experimental Analysis of In-Pipe Image Transmission Based on Visible Light Relay Communication.

The regular inspection of underground facilities such as pipelines is absolutely essential. Pipeline leakage caused by corrosion and deformation must be detected in time, otherwise, it may cause fatal disasters for human beings. In our previous research, a robot chain system (RCS) based on visible light relay communication (VLRC) for pipe inspection has been developed. This system can basically realize the light-based transmission of control command signals and illuminance-based coordinated movement, whereas the collection and transmission approach of the pipe leakage image have not been studied. Compared with former in-pipe wireless communication techniques, VLRC can not only overcome the instability and inefficiency of in-pipe data transmission but also extend the communication range with high transmission rates. The most important feature is that it can provide a stable illumination and high-quality communication for pipe inspection robot and finally improve the energy efficiency. Hence, the aim of this article is to analyze the performance of VLRC-based image transmission in the pipe and in the future provide a high-quality, long-range, and high-efficiency image transmission for complex infrastructure inspection with RCS. The transmission systems based on two signal transmission modes analog image signal relay transmission (AISRT) and digital image frame relay transmission (DIFRT) have been proposed. Multiple experiments including the waveform test, the test of transmission features with different bit error rate (BER), and in the different mediums were conducted between these two systems. The experiment revealed that DIFRT was superior to the AISRT in terms of the relatively high-quality image transmission and reconstruction quality. It could better overcome the attenuation brought by the absorption and scattering effects and finally increased the transmission range than former communication methods. The DIFRT system could also operate at 50 kbps with relatively low BER whether in the air or water. The technique in this research could potentially provide a new strategy for implementations in the stable, effective, high-speed, and long-range image transmission of the robots in some other special environments such as tunnel, mine, and underwater, etc.

Prediction Algorithm of Parameters of Toe Clearance in the Swing Phase.

The adaptive control of gait training robots is aimed at improving the gait performance by assisting motion. In conventional robotics, it has not been possible to adjust the robotic parameters by predicting the toe motion, which is considered a tripping risk indicator. The prediction of toe clearance during walking can decrease the risk of tripping. In this paper, we propose a novel method of predicting toe clearance that uses a radial basis function network. The input data were the angles, angular velocities, and angular accelerations of the hip, knee, and ankle joints in the sagittal plane at the beginning of the swing phase. In the experiments, seven subjects walked on a treadmill for 360 s. The radial basis function network was trained with gait data ranging from 20 to 200 data points and tested with 100 data points. The root mean square error between the true and predicted values was 3.28 mm for the maximum toe clearance in the earlier swing phase and 2.30 mm for the minimum toe clearance in the later swing phase. Moreover, using gait data of other five subjects, the root mean square error between the true and predicted values was 4.04 mm for the maximum toe clearance and 2.88 mm for the minimum toe clearance when the walking velocity changed. This provided higher prediction accuracy compared with existing methods. The proposed algorithm used the information of joint movements at the start of the swing phase and could predict both the future maximum and minimum toe clearances within the same swing phase.

Soft Magnetic Powdery Sensor for Tactile Sensing.

Soft resistive tactile sensors are versatile devices with applications in next-generation flexible electronics. We developed a novel type of soft resistive tactile sensor called a soft magnetic powdery sensor (soft-MPS) and evaluated its response characteristics. The soft-MPS comprises ferromagnetic powder that is immobilized in a liquid resin such as polydimethylsiloxane (PDMS) after orienting in a magnetic field. On applying an external force to the sensor, the relative distance between particles changes, thereby affecting its resistance. Since the ferromagnetic powders are in contact from the initial state, they have the ability to detect small contact forces compared to conventional resistive sensors in which the conductive powder is dispersed in a flexible material. The sensor unit can be made in any shape by controlling the layout of the magnetic field. Soft-MPSs with different hardnesses that could detect small forces were fabricated. The soft-MPS could be applied to detect collisions in robot hands/arms or in ultra-sensitive touchscreen devices.

A Coordinated Wheeled Gas Pipeline Robot Chain System Based on Visible Light Relay Communication and Illuminance Assessment.

The gas pipeline requires regular inspection since the leakage brings damage to the stable gas supply. Compared to current detection methods such as destructive inspection, using pipeline robots has advantages including low cost and high efficiency. However, they have a limited inspection range in the complex pipe owing to restrictions by the cable friction or wireless signal attenuation. In our former study, to extend the inspection range, we proposed a robot chain system based on wireless relay communication (WRC). However, some drawbacks still remain such as imprecision of evaluation based on received signal strength indication (RSSI), large data error ratio, and loss of signals. In this article, we thus propose a new approach based on visible light relay communication (VLRC) and illuminance assessment. This method enables robots to communicate by the 'light signal relay', which has advantages in good communication quality, less attenuation, and high precision in the pipe. To ensure the stability of VLRC, the illuminance-based evaluation method is adopted due to higher stability than the wireless-based approach. As a preliminary evaluation, several tests about signal waveform, communication quality, and coordinated movement were conducted. The results indicate that the proposed system can extend the inspection range with less data error ratio and more stable communication.

Prediction of Minimum Toe Clearance with a Radial Basis Function Network at the Start of the Swing Phase.

Prediction of minimum toe clearance (MTC) during walking can decrease the risk of tripping. In this paper, we proposed a novel MTC prediction method using a radial basis function network. Input data were the angles, angular velocities, and angular accelerations of the hip, knee, and ankle joints in the sagittal plane at the start of the swing phase. In experiments, five subjects walked on a treadmill for 360 s. The radial basis function network was trained with 60 s of gait data and tested with the remaining 300 s of gait data. The root mean square error between the true and predicted MTC values was lower than 2.79 mm in all subjects.

Evaluation of Compensatory Movement by Shoulder Joint Torque during Gain Adjustment of a Powered Prosthetic Wrist Joint.

Powered prostheses with low degree of freedom (DoF) have been developed for people with disabilities to assist daily tasks. These prostheses neglect the user's compensatory movements caused by the low degree of freedom. We assume that the movements can be reduced by well-designed controller of the devices. This paper explores an optimal control gain of the powered prosthesis to prevent the user from compensatory movements through experiments. In the experiments, we developed 1-DoF hand prosthesis with a position-controlled servo, which includes the constant gain as a feed-forward term. The compensatory movements are regarded as a joint torque at a shoulder (abduction/adduction). 4 intact subjects performed a pick-and-place task, using the prosthesis with several control gains. The empirical results show that there was the optimal gain for each subject, which reduces their compensatory movement.

Timing of intermittent torque control with wire-driven gait training robot lifting toe trajectory for trip avoidance.

Gait training robots are useful for changing gait patterns and decreasing risk of trip. Previous research has reported that decreasing duration of the assistance or guidance of the robot is beneficial for efficient gait training. Although robotic intermittent control method for assisting joint motion has been established, the effect of the robot intervention timing on change of toe clearance is unclear. In this paper, we tested different timings of applying torque to the knee, employing the intermittent control of a gait training robot to increase toe clearance throughout the swing phase. We focused on knee flexion motion and designed a gait training robot that can apply flexion torque to the knee with a wire-driven system. We used a method of timing detecting for the robot conducting torque control based on information from the hip, knee, and ankle angles to establish a non-time dependent parameter that can be used to adapt to gait change, such as gait speed. We carried out an experiment in which the conditions were four time points: starting the swing phase, lifting the foot, maintaining knee flexion, and finishing knee flexion. The results show that applying flexion torque to the knee at the time point when people start lifting their toe is effective for increasing toe clearance in the whole swing phase.

Change detection technique for muscle tone during static stretching by continuous muscle viscoelasticity monitoring using wearable indentation tester.

Static stretching is widely performed to decrease muscle tone as a part of rehabilitation protocols. Finding out the optimal duration of static stretching is important to minimize the time required for rehabilitation therapy and it would be helpful for maintaining the patient's motivation towards daily rehabilitation tasks. Several studies have been conducted for the evaluation of static stretching; however, the recommended duration of static stretching varies widely between 15-30 s in general, because the traditional methods for the assessment of muscle tone do not monitor the continuous change in the target muscle's state. We have developed a method to monitor the viscoelasticity of one muscle continuously during static stretching, using a wearable indentation tester. In this study, we investigated a suitable signal processing method to detect the time required to change the muscle tone, utilizing the data collected using a wearable indentation tester. By calculating a viscoelastic index with a certain time window, we confirmed that the stretching duration required to bring about a decrease in muscle tone could be obtained with an accuracy in the order of 1 s.

Learning to Perceive the World as Probabilistic or Deterministic via Interaction With Others: A Neuro-Robotics Experiment.

We suggest that different behavior generation schemes, such as sensory reflex behavior and intentional proactive behavior, can be developed by a newly proposed dynamic neural network model, named stochastic multiple timescale recurrent neural network (S-MTRNN). The model learns to predict subsequent sensory inputs, generating both their means and their uncertainty levels in terms of variance (or inverse precision) by utilizing its multiple timescale property. This model was employed in robotics learning experiments in which one robot controlled by the S-MTRNN was required to interact with another robot under the condition of uncertainty about the other's behavior. The experimental results show that self-organized and sensory reflex behavior-based on probabilistic prediction-emerges when learning proceeds without a precise specification of initial conditions. In contrast, intentional proactive behavior with deterministic predictions emerges when precise initial conditions are available. The results also showed that, in situations where unanticipated behavior of the other robot was perceived, the behavioral context was revised adequately by adaptation of the internal neural dynamics to respond to sensory inputs during sensory reflex behavior generation. On the other hand, during intentional proactive behavior generation, an error regression scheme by which the internal neural activity was modified in the direction of minimizing prediction errors was needed for adequately revising the behavioral context. These results indicate that two different ways of treating uncertainty about perceptual events in learning, namely, probabilistic modeling and deterministic modeling, contribute to the development of different dynamic neuronal structures governing the two types of behavior generation schemes.

Design and Characterization of a Three-Axis Hall Effect-Based Soft Skin Sensor.

This paper presents an easy means to produce a 3-axis Hall effect-based skin sensor for robotic applications. It uses an off-the-shelf chip and is physically small and provides digital output. Furthermore, the sensor has a soft exterior for safe interactions with the environment; in particular it uses soft silicone with about an 8 mm thickness. Tests were performed to evaluate the drift due to temperature changes, and a compensation using the integral temperature sensor was implemented. Furthermore, the hysteresis and the crosstalk between the 3-axis measurements were evaluated. The sensor is able to detect minimal forces of about 1 gf. The sensor was calibrated and results with total forces up to 1450 gf in the normal and tangential directions of the sensor are presented. The test revealed that the sensor is able to measure the different components of the force vector.

Improving IMES Localization Accuracy by Integrating Dead Reckoning Information.

Indoor positioning remains an open problem, because it is difficult to achieve satisfactory accuracy within an indoor environment using current radio-based localization technology. In this study, we investigate the use of Indoor Messaging System (IMES) radio for high-accuracy indoor positioning. A hybrid positioning method combining IMES radio strength information and pedestrian dead reckoning information is proposed in order to improve IMES localization accuracy. For understanding the carrier noise ratio versus distance relation for IMES radio, the signal propagation of IMES radio is modeled and identified. Then, trilateration and extended Kalman filtering methods using the radio propagation model are developed for position estimation. These methods are evaluated through robot localization and pedestrian localization experiments. The experimental results show that the proposed hybrid positioning method achieved average estimation errors of 217 and 1846 mm in robot localization and pedestrian localization, respectively. In addition, in order to examine the reason for the positioning accuracy of pedestrian localization being much lower than that of robot localization, the influence of the human body on the radio propagation is experimentally evaluated. The result suggests that the influence of the human body can be modeled.

Relationship between magnitude of applied torque in pre-swing phase and gait change for prevention of trip in elderly people.

Elderly people are at risk of tripping because of their narrow range of articular motion. To avoid tripping, gait training that improves their range of articular motion would be beneficial. In this study we propose a gait-training robot that applies a torque during the pre-swing phase to achieve this goal. We investigated the relationship between magnitude of applied torque and change in the range of knee-articular motion while walking before and after the application of this torque. We developed a wearable robot and carried out an experiment on human participants in which a motor pulls a string embedded on the robotic frame, applying torque in the pre-swing phase for a period of 20 [s]. Before and after applying torque the participant walked normally for 15 [s] without interference from the robot. We found that knee flexion angle increased after applying the torque if the torque was within the range of approximately 6-8 [Nm]. Therefore, we were able to verify that a new range of knee articular motion can be learned through application of torque.